If Providing Best Care Means Being at the Cutting Edge of Research, Should It Be Implemented System-wide? Comment on "'We're Not Providing the Best Care If We Are Not on the Cutting Edge of Research': A Research Impact Evaluation at a Regional Australian Hospital and Health Service".

The COVID-19 pandemic intensified debates about the desirability of integrating health research systems into healthcare systems. An excellent evaluation undertaken prior to the pandemic examined a purposeful strategy to improve healthcare through an expansion in research capacity in the Townsville Hospital and Health Service (THHS), a regional service in northern Queensland. This comment puts that evaluation into a rapidly developing wider context, drawing on other work showing an association between research engagement in healthcare organisations and their improved healthcare. In most previous studies this impact arose as a by-product of the research activity. The Townsville scheme went further. But while the evaluation identified some progress and impacts, they were patchy, not system-wide. Recent pre-pandemic studies showed that going even further and integrating a health research system across a national healthcare system markedly improved healthcare, despite continuing challenges. The UK's research experiences during COVID-19 are giving additional momentum to this approach globally.

Saving millions of lives but some resources squandered: emerging lessons from health research system pandemic achievements and challenges.

During the SARS-CoV-2 pandemic, astonishingly rapid research averted millions of deaths worldwide through new vaccines and repurposed and new drugs. Evidence use informed life-saving national policies including non-pharmaceutical interventions. Simultaneously, there was unprecedented waste, with many underpowered trials on the same drugs. We identified lessons from COVID-19 research responses by applying WHO's framework for research systems. It has four functions-governance, securing finance, capacity-building, and production and use of research-and nine components. Two linked questions focused the analysis. First, to what extent have achievements in knowledge production and evidence use built on existing structures and capacity in national health research systems? Second, did the features of such systems mitigate waste? We collated evidence on seven countries, Australia, Brazil, Canada, Germany, New Zealand, the United Kingdom and the United States, to identify examples of achievements and challenges.We used the data to develop lessons for each framework component. Research coordination, prioritization and expedited ethics approval contributed to rapid identification of new therapies, including dexamethasone in the United Kingdom and Brazil. Accelerated vaccines depended on extensive funding, especially through the Operation Warp Speed initiative in the United States, and new platforms created through long-term biomedical research capacity in the United Kingdom and, for messenger ribonucleic acid (mRNA) vaccines, in Canada, Germany and the United States. Research capacity embedded in the United Kingdom's healthcare system resulted in trial acceleration and waste avoidance. Faster publication of research saved lives, but raised challenges. Public/private collaborations made major contributions to vastly accelerating new products, available worldwide, though unequally. Effective developments of living (i.e. regularly updated) reviews and guidelines, especially in Australia and Canada, extended existing expertise in meeting users' needs. Despite complexities, effective national policy responses (less evident in Brazil, the United Kingdom and the United States) also saved lives by drawing on health research system features, including collaboration among politicians, civil servants and researchers; good communications; and willingness to use evidence. Comprehensive health research strategies contributed to success in research production in the United Kingdom and in evidence use by political leadership in New Zealand. In addition to waste, challenges included equity issues, public involvement and non-COVID research. We developed recommendations, but advocate studies of further countries.

A systems approach for optimizing implementation to impact: meeting report and proceedings of the 2019 In the Trenches: Implementation to Impact International Summit.

BACKGROUND: The In the Trenches series of cutting-edge knowledge sharing events on impact for front-line experts and practitioners provides an engagement platform for diverse stakeholders across government, research funding organizations, industry, and academia to share emerging knowledge and practical experiences. The second event of the series In the Trenches: Implementation to Impact International Summit was held in Banff, Alberta, Canada, on June 7-8, 2019. The overarching vision for the Summit was to create an engagement platform for addressing key challenges and finding practical solutions to move from implementation (i.e. putting findings into effect) to impact (i.e. creating benefits to society and the economy). PROCESSES AND PROCEEDINGS: The Summit used diverse approaches to facilitate active engagement and knowledge sharing between 80 delegates across sectors and jurisdictions. Summit sessions mostly consisted of short talks and moderated panels grouped into eight thematic sessions. Each presentation included a summary of Key Messages, along with a summary of the Actionable Insights which concluded each session. The presentations and discussions are analysed, synthesized and described in this proceedings paper using a systems approach. This demonstrates how the Summit focused on each of the necessary functions (and associated components) that should be undertaken, and combined, for effective research and innovation: stewardship and governance, securing finance, creating capacity, and producing and using research. The approach also identifies relevant challenges. CONCLUSIONS: There is increased interest globally in the benefits that can accrue from adopting a systems approach to research and innovation. Various organizations in Canada and internationally have made considerable progress on Implementation to Impact, often as a result of well-planned initiatives. The Summit highlights the value of 1) collaboration between researchers and potential users, and 2) the adoption by funders of approaches involving an increasing range of responsibilities and activities. The Summit website (https://inthetrenchessummit.com/) will be periodically updated with new resources and information about future In the Trenches events.

From COVID-19 research to vaccine application: why might it take 17 months not 17 years and what are the wider lessons?

It is often said that it takes 17 years to move medical research from bench to bedside. In a coronavirus disease (COVID-19) world, such time-lags feel intolerable. In these extraordinary circumstances could years be made into months? If so, could those lessons be used to accelerate medical research when the crisis eases?To measure time-lags in health and biomedical research as well as to identify ways of reducing them, we developed and published (in 2015) a matrix consisting of overlapping tracks (or stages/phases) in the translation from discovery research to developed products, policies and practice. The matrix aids analysis by highlighting the time and actions required to develop research (and its translation) both (1) along each track and (2) from one track to another, e.g. from the discovery track to the research-in-humans track. We noted four main approaches to reducing time-lags, namely increasing resources, working in parallel, starting or working at risk, and improving processes.Examining these approaches alongside the matrix helps interpret the enormous global effort to develop a vaccine for the 2019 novel coronavirus SARS-CoV-2, the causative agent of COVID-19. Rapid progress in the discovery/basic and human research tracks is being made through a combination of large-scale funding, work being conducted in parallel (between different teams globally and through working in overlapping tracks), working at greater (but proportionate) risk to safety than usual, and adopting various new processes. The overlapping work of some of the teams involves continuing animal research whilst entering vaccine candidates into Phase I trials alongside planning their Phase II trials. The additional funding available helps to reduce some of the usual financial risks in moving so quickly. Going forward through the increasingly large human trials for safety, dosage and efficacy, it will be vital to overlap work in parallel in the often challenging public policy and clinical tracks. Thus, regulatory and reimbursement bodies are beginning and preparing rapid action to pull vaccines proving to be safe and effective through to extraordinarily rapid application to the general population. Monitoring the development of a COVID-19 vaccine using the matrix (modified as necessary) could help identify which of the approaches speeding development and deployment could be usefully applied more widely in the future.

How to strengthen a health research system: WHO's review, whose literature and who is providing leadership?

BACKGROUND: Health research is important for the achievement of the Sustainable Development Goals. However, there are many challenges facing health research, including securing sufficient funds, building capacity, producing research findings and using both local and global evidence, and avoiding waste. A WHO initiative addressed these challenges by developing a conceptual framework with four functions to guide the development of national health research systems. Despite some progress, more is needed before health research systems can meet their full potential of improving health systems. The WHO Regional Office for Europe commissioned an evidence synthesis of the systems-level literature. This Opinion piece considers its findings before reflecting on the vast additional literature available on the range of specific health research system functions related to the various challenges. Finally, it considers who should lead research system strengthening. MAIN TEXT: The evidence synthesis identifies two main approaches for strengthening national health research systems, namely implementing comprehensive and coherent strategies and participation in partnerships. The literature describing these approaches at the systems level also provides data on ways to strengthen each of the four functions of governance, securing financing, capacity-building, and production and use of research. Countries effectively implementing strategies include England, Ireland and Rwanda, whereas West Africa experienced effective partnerships. Recommended policy approaches for system strengthening are context specific. The vast literature on each function and the ever-growing evidence-base are illustrated by considering papers in just one key journal, Health Research Policy and Systems, and analysing the contribution of two national studies. A review of the functions of the Iranian system identifies over 200 relevant and mostly national records; an analysis of the creation of the English National Institute for Health Research describes the key leadership role played by the health department. Furthermore, WHO is playing leadership roles in helping coordinate partnerships within and across health research systems that have been attempting to tackle the COVID-19 crisis. CONCLUSIONS: The evidence synthesis provides a firm basis for decision-making by policy-makers and research leaders looking to strengthen national health research systems within their own national context. It identifies five crucial policy approaches - conducting situation analysis, sustaining a comprehensive strategy, engaging stakeholders, evaluating impacts on health systems, and partnership participation. The vast and ever-growing additional literature could provide further perspectives, including on crucial leadership roles for health ministries.

'Knowledge for better health' revisited - the increasing significance of health research systems: a review by departing Editors-in-Chief.

How can nations organise research investments to obtain the best bundle of knowledge and the maximum level of improved health, spread as equitably as possible? This question was the central focus of a major initiative from WHO led by Prof Tikki Pang, which resulted in a range of developments, including the publication of a conceptual framework for national health research systems - Knowledge for better health - in 2003, and in the founding of the journal Health Research Policy and Systems (HARPS). As Editors-in-Chief of the journal since 2006, we mark our retirement by tracking both the progress of the journal and the development of national health research systems. HARPS has maintained its focus on a range of central themes that are key components of a national health research system in any country. These include building capacity to conduct and use health research, identifying appropriate priorities, securing funds and allocating them accountably, producing scientifically valid research outputs, promoting the use of research in polices and practice in order to improve health, and monitoring and evaluating the health research system. Some of the themes covered in HARPS are now receiving increased attention and, for example, with the assessment of research impact and development of knowledge translation platforms, the journal has covered their progress throughout that expansion of interest. In addition, there is increasing recognition of new imperatives, including the importance of promoting gender equality in health research if benefits are to be maximised. In this Editorial, we outline some of the diverse and developing perspectives considered within each theme, as well as considering how they are held together by the growing desire to build effective health research systems in all countries.From 2003 until mid-June 2017, HARPS published 590 articles on the above and related themes, with authors being located in 76 countries. We present quantitative data tracing the journal's growth and the increasing external recognition of its role. We thank the many colleagues who have kindly contributed to the journal's success, and finish on an exciting note by welcoming the new Editors-in-Chief who will take HARPS forward.

Building health research systems: WHO is generating global perspectives, and who's celebrating national successes?

In 2016, England's National Institute for Health Research (NIHR) celebrated its tenth anniversary as an innovative national health research system with a focus on meeting patients' needs. This provides a good opportunity to reflect on how the creation of the NIHR has greatly enhanced important work, started in 1991, to develop a health research system in England that is embedded in the National Health Service.In 2004, WHO identified a range of functions that a national health research system should undertake to improve the health of populations. Health Research Policy and Systems (HRPS) has taken particular interest in the pioneering developments in the English health research system, where the comprehensive approach has covered most, if not all, of the functions identified by WHO. Furthermore, several significant recent developments in thinking about health research are relevant for the NIHR and have informed accounts of its achievements. These include recognition of the need to combat waste in health research, which had been identified as a global problem in successive papers in the Lancet, and an increasing emphasis on demonstrating impact. Here, pioneering evaluation of United Kingdom research, conducted through the impact case studies of the Research Excellence Framework, is particularly important. Analyses informed by these and other approaches identified many aspects of NIHR's progress in combating waste, building and sustaining research capacity, creating centres of research excellence linked to leading healthcare institutions, developing research networks, involving patients and others in identifying research needs, and producing and adopting research findings that are improving health outcomes.The NIHR's overall success, and an analysis of the remaining problems, might have lessons for other systems, notwithstanding important advances in many countries, as described in papers in HRPS and elsewhere. WHO's recently established Global Observatory for Health Research and Development provides an opportunity to promote some of these lessons. To inform its work, the Observatory is sponsoring a thematic series of papers in HRPS focusing on health research issues such as funding flows, priority setting, capacity building, utilisation and equity. While important papers on these have been published, this series is still open to new submissions.

Health research improves healthcare: now we have the evidence and the chance to help the WHO spread such benefits globally.

There has been a dramatic increase in the body of evidence demonstrating the benefits that come from health research. In 2014, the funding bodies for higher education in the UK conducted an assessment of research using an approach termed the Research Excellence Framework (REF). As one element of the REF, universities and medical schools in the UK submitted 1,621 case studies claiming to show the impact of their health and other life sciences research conducted over the last 20 years. The recently published results show many case studies were judged positively as providing examples of the wide range and extensive nature of the benefits from such research, including the development of new treatments and screening programmes that resulted in considerable reductions in mortality and morbidity.Analysis of specific case studies yet again illustrates the international dimension of progress in health research; however, as has also long been argued, not all populations fully share the benefits. In recognition of this, in May 2013 the World Health Assembly requested the World Health Organization (WHO) to establish a Global Observatory on Health Research and Development (R&D) as part of a strategic work-plan to promote innovation, build capacity, improve access, and mobilise resources to address diseases that disproportionately affect the world's poorest countries.As editors of Health Research Policy and Systems (HARPS), we are delighted that our journal has been invited to help inform the establishment of the WHO Global Observatory through a Call for Papers covering a range of topics relevant to the Observatory, including topics on which HARPS has published articles over the last few months, such as approaches to assessing research results, measuring expenditure data with a focus on R&D, and landscape analyses of platforms for implementing R&D. Topics related to research capacity building may also be considered. The task of establishing a Global Observatory on Health R&D to achieve the specified objectives will not be easy; nevertheless, this Call for Papers is well timed - it comes just at the point where the evidence of the benefits from health research has been considerably strengthened.

How long does biomedical research take? Studying the time taken between biomedical and health research and its translation into products, policy, and practice.

BACKGROUND: The time taken, or 'time lags', between biomedical/health research and its translation into health improvements is receiving growing attention. Reducing time lags should increase rates of return to such research. However, ways to measure time lags are under-developed, with little attention on where time lags arise within overall timelines. The process marker model has been proposed as a better way forward than the current focus on an increasingly complex series of translation 'gaps'. Starting from that model, we aimed to develop better methods to measure and understand time lags and develop ways to identify policy options and produce recommendations for future studies. METHODS: Following reviews of the literature on time lags and of relevant policy documents, we developed a new approach to conduct case studies of time lags. We built on the process marker model, including developing a matrix with a series of overlapping tracks to allow us to present and measure elements within any overall time lag. We identified a reduced number of key markers or calibration points and tested our new approach in seven case studies of research leading to interventions in cardiovascular disease and mental health. Finally, we analysed the data to address our study's key aims. RESULTS: The literature review illustrated the lack of agreement on starting points for measuring time lags. We mapped points from policy documents onto our matrix and thus highlighted key areas of concern, for example around delays before new therapies become widely available. Our seven completed case studies demonstrate we have made considerable progress in developing methods to measure and understand time lags. The matrix of overlapping tracks of activity in the research and implementation processes facilitated analysis of time lags along each track, and at the cross-over points where the next track started. We identified some factors that speed up translation through the actions of companies, researchers, funders, policymakers, and regulators. Recommendations for further work are built on progress made, limitations identified and revised terminology. CONCLUSIONS: Our advances identify complexities, provide a firm basis for further methodological work along and between tracks, and begin to indicate potential ways of reducing lags.

Four centuries on from Bacon: progress in building health research systems to improve health systems?

In 1627, Francis Bacon's New Atlantis described a utopian society in which an embryonic research system contributed to meeting the needs of the society. In this editorial, we use some of the aspirations described in New Atlantis to provide a context within which to consider recent progress in building health research systems to improve health systems and population health. In particular, we reflect on efforts to build research capacity, link research to policy, identify the wider impacts made by the science, and generally build fully functioning research systems to address the needs identified. In 2014, Health Research Policy and Systems has continued to publish one-off papers and article collections covering a range of these issues in both high income countries and low- and middle-income countries. Analysis of these contributions, in the context of some earlier ones, is brought together to identify achievements, challenges and possible ways forward. We show how 2014 is likely to be a pivotal year in the development of ways to assess the impact of health research on policies, practice, health systems, population health, and economic benefits.We demonstrate how the increasing focus on health research systems will contribute to realising the hopes expressed in the World Health Report, 2013, namely that all nations would take a systematic approach to evaluating the outputs and applications resulting from their research investment.

Understanding factors associated with the translation of cardiovascular research: a multinational case study approach.

BACKGROUND: Funders of health research increasingly seek to understand how best to allocate resources in order to achieve maximum value from their funding. We built an international consortium and developed a multinational case study approach to assess benefits arising from health research. We used that to facilitate analysis of factors in the production of research that might be associated with translating research findings into wider impacts, and the complexities involved. METHODS: We built on the Payback Framework and expanded its application through conducting co-ordinated case studies on the payback from cardiovascular and stroke research in Australia, Canada and the United Kingdom. We selected a stratified random sample of projects from leading medical research funders. We devised a series of innovative steps to: minimize the effect of researcher bias; rate the level of impacts identified in the case studies; and interrogate case study narratives to identify factors that correlated with achieving high or low levels of impact. RESULTS: Twenty-nine detailed case studies produced many and diverse impacts. Over the 15 to 20 years examined, basic biomedical research has a greater impact than clinical research in terms of academic impacts such as knowledge production and research capacity building. Clinical research has greater levels of wider impact on health policies, practice, and generating health gains. There was no correlation between knowledge production and wider impacts. We identified various factors associated with high impact. Interaction between researchers and practitioners and the public is associated with achieving high academic impact and translation into wider impacts, as is basic research conducted with a clinical focus. Strategic thinking by clinical researchers, in terms of thinking through pathways by which research could potentially be translated into practice, is associated with high wider impact. Finally, we identified the complexity of factors behind research translation that can arise in a single case. CONCLUSIONS: We can systematically assess research impacts and use the findings to promote translation. Research funders can justify funding research of diverse types, but they should not assume academic impacts are proxies for wider impacts. They should encourage researchers to consider pathways towards impact and engage potential research users in research processes.

Evaluation of the impact of National Breast Cancer Foundation-funded research.

OBJECTIVE: To evaluate the impact of the National Breast Cancer Foundation's (NBCF's) research investment. DESIGN AND PARTICIPANTS: Surveys based on the Payback Framework were sent to chief investigators involved in research funded by the NBCF during 1995-2012; a bibliometric analysis of NBCF-funded publications in 2006-2010 was conducted; and a purposive, stratified sample of case studies was obtained. MAIN OUTCOME MEASURES: Research impact on knowledge production, the research system, informing policy, product development and broader health and economic benefits. RESULTS: Of 242 surveys sent, 153 (63%) were returned. The average impact of journals in which NBCF publications appeared was double that of world publications. Seventy surveys (46%) reported career progression, and 185 higher degrees were obtained or expected, including 121 PhDs. One hundred and one grants (66%) produced tools that built capacity across the research system, and research teams leveraged an additional $1.40 in funding for every dollar invested. Fifteen applied grants and one basic grant impacted on policy. Ten basic and four applied grants led to the development of drugs, prognostic tools or diagnostic technologies. Twenty applied and two basic grants led to changes in practice and behaviour of health care staff, consumers and the public, with further impacts anticipated. Case studies provided illustrations of high impact. CONCLUSIONS: NBCF's strategy of investing in a mixed portfolio of research areas and mechanisms encouraged a broad range of impacts across all Payback categories. The impacts from basic research tended to focus on knowledge production and drug development; while applied research generated greater impacts within the other Payback categories. The funding of shared infrastructure stimulated impact across the research system.

Organising health research systems as a key to improving health: the World Health Report 2013 and how to make further progress.

The World Health Report 2013 provides a major boost to the health research community and, in particular, to those who believe that health research will make its greatest impact on improving health when it is organised through a systems approach. The World Health Report 2013, Research for Universal Health Coverage, starts with three key messages. Firstly, that universal health coverage, with full access to high-quality services, needs research evidence if it is to be achieved; second, all nations should conduct and use research; and finally, the report states that systems are needed to develop national research agendas, to raise funds, to strengthen research capacity, and to make effective use of research findings. Each of these themes is elaborated in the report and supported by extensive references.In this editorial, we first outline the key messages from the World Health Report 2013 and highlight the contributions made by papers from our journal, Health Research Policy and Systems. In addition, we discuss very recent papers that advance some issues even further. In particular, we consider new evidence both on how to achieve financial protection for those who use health services, and on whether healthcare professionals and organisations who engage in research provide an improved healthcare performance. Finally, we propose additional perspectives that add to the impressive body of evidence and analyses presented in the report. Specifically, we suggest that considering the needs of various stakeholders, as attempted in the UK, in parallel with analysing how to fulfil essential functions, should boost the prospects of successfully building and strengthening health research systems. This is important because research is vital for achieving universal health coverage, and consequently for improving the health of millions of people.

Conducting retrospective impact analysis to inform a medical research charity's funding strategies: the case of Asthma UK.

BACKGROUND: Debate is intensifying about how to assess the full range of impacts from medical research. Complexity increases when assessing the diverse funding streams of funders such as Asthma UK, a charitable patient organisation supporting medical research to benefit people with asthma. This paper aims to describe the various impacts identified from a range of Asthma UK research, and explore how Asthma UK utilised the characteristics of successful funding approaches to inform future research strategies. METHODS: We adapted the Payback Framework, using it both in a survey and to help structure interviews, documentary analysis, and case studies. We sent surveys to 153 lead researchers of projects, plus 10 past research fellows, and also conducted 14 detailed case studies. These covered nine projects and two fellowships, in addition to the innovative case studies on the professorial chairs (funded since 1988) and the MRC-Asthma UK Centre in Allergic Mechanisms of Asthma (the 'Centre') which together facilitated a comprehensive analysis of the whole funding portfolio. We organised each case study to capture whatever academic and wider societal impacts (or payback) might have arisen given the diverse timescales, size of funding involved, and extent to which Asthma UK funding contributed to the impacts. RESULTS: Projects recorded an average of four peer-reviewed journal articles. Together the chairs reported over 500 papers. All streams of funding attracted follow-on funding. Each of the various categories of societal impacts arose from only a minority of individual projects and fellowships. Some of the research portfolio is influencing asthma-related clinical guidelines, and some contributing to product development. The latter includes potentially major breakthroughs in asthma therapies (in immunotherapy, and new inhaled drugs) trialled by university spin-out companies. Such research-informed guidelines and medicines can, in turn, contribute to health improvements. The role of the chairs and the pioneering collaborative Centre is shown as being particularly important. CONCLUSIONS: We systematically demonstrate that all types of Asthma UK's research funding assessed are making impacts at different levels, but the main societal impacts from projects and fellowships come from a minority of those funded. Asthma UK used the study's findings, especially in relation to the Centre, to inform research funding strategies to promote the achievement of impact.

Assessing the impact of health technology assessment in The Netherlands.

OBJECTIVES: Investments in health research should lead to improvements in health and health care. This is also the remit of the main HTA program in the Netherlands. The aims of this study were to assess whether the results of this program have led to such improvements and to analyze how best to assess the impact from health research. METHODS: We assessed the impact of individual HTA projects by adapting the "payback framework" developed in the United Kingdom. We conducted dossier reviews and sent a survey to principal investigators of forty-three projects awarded between 2000 and 2003. We then provided an overview of documented output and outcome that was assessed by ten HTA experts using a scoring method. Finally, we conducted five case studies using information from additional dossier review and semistructured key informant interviews. RESULTS: The findings confirm that the payback framework is a useful approach to assess the impact of HTA projects. We identified over 101 peer reviewed papers, more than twenty-five PhDs, citations of research in guidelines (six projects), and implementation of new treatment strategies (eleven projects). The case studies provided greater depth and understanding about the levels of impact that arise and why and how they have been achieved. CONCLUSIONS: It is generally too early to determine whether the HTA program led to actual changes in healthcare policy and practice. However, the results can be used as a baseline measurement for future evaluation and can help funding organizations or HTA agencies consider how to assess impact, possibly routinely. This, in turn, could help inform research strategies and justify expenditure for health research.

Lessons from the evaluation of the UK's NHS R&D implementation methods programme.

BACKGROUND: Concern about the effective use of research was a major factor behind the creation of the NHS R&D Programme in 1991. In 1994, an advisory group was established to identify research priorities in research implementation. The Implementation Methods Programme (IMP) flowed from this, and its commissioning group funded 36 projects. In 2000 responsibility for the programme passed to the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D, which asked the Health Economics Research Group (HERG), Brunel University, to conduct an evaluation in 2002. By then most projects had been completed. This evaluation was intended to cover: the quality of outputs, lessons to be learnt about the communication strategy and the commissioning process, and the benefits from the projects. METHODS: We adopted a wide range of quantitative and qualitative methods. They included: documentary analysis, interviews with key actors, questionnaires to the funded lead researchers, questionnaires to potential users, and desk analysis. RESULTS: Quantitative assessment of outputs and dissemination revealed that the IMP funded useful research projects, some of which had considerable impact against the various categories in the HERG payback model, such as publications, further research, research training, impact on health policy, and clinical practice. Qualitative findings from interviews with advisory and commissioning group members indicated that when the IMP was established, implementation research was a relatively unexplored field. This was reflected in the understanding brought to their roles by members of the advisory and commissioning groups, in the way priorities for research were chosen and developed, and in how the research projects were commissioned. The ideological and methodological debates associated with these decisions have continued among those working in this field. The need for an effective communication strategy for the programme as a whole was particularly important. However, such a strategy was never developed, making it difficult to establish the general influence of the IMP as a programme. CONCLUSION: Our findings about the impact of the work funded, and the difficulties faced by those developing the IMP, have implications for the development of strategic programmes of research in general, as well as for the development of more effective research in this field.